{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "tuOe1ymfHZPu"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9vfDV6DhVOAj"
      },
      "source": [
        "# Colab Magic\n",
        "\n",
        "This notebook introduces Colab magic commands for PaLM. Magics make it easy to develop, test, compare, and evaluate prompts from within a Colab notebook."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1d0c4d7036cc"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://ai.google.dev/palm_docs/notebook_magic\"><img src=\"https://ai.google.dev/static/site-assets/images/docs/notebook-site-button.png\" height=\"32\" width=\"32\" />View on ai.google.dev</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/palm_docs/notebook_magic.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/google/generative-ai-docs/blob/main/site/en/palm_docs/notebook_magic.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EkqWcXtIVY63"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oZQhkjBevAA8"
      },
      "source": [
        "Follow the steps below to install and test the magics.\n",
        "\n",
        "### Installing the PaLM magic\n",
        "To use the PaLM magic commands in Colab or other IPython environment, you will first need to download and install the [`google-generativeai`](https://pypi.org/project/google-generativeai) Python package."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4so2CHmPSxIU"
      },
      "outputs": [],
      "source": [
        "%pip install -q google-generativeai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "84yKJHcyWLSA"
      },
      "source": [
        "### Loading the PaLM magic\n",
        "\n",
        "Next, load the `%%palm` magic by using the `%load_ext` magic:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3qoE1eycyJzD"
      },
      "outputs": [],
      "source": [
        "%load_ext google.generativeai.notebook"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1cLrcdpF6avu"
      },
      "source": [
        "### Test the installation\n",
        "To test for correct installation of the magic commands, run `%%palm --help`. Note that you will also need a PaLM API key, if you don't have one already (see next step)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UKkEPjKKU9o2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "usage: palm [-h] {run,compile,compare,eval} ...\n",
            "\n",
            "A system for interacting with LLMs.\n",
            "\n",
            "positional arguments:\n",
            "  {run,compile,compare,eval}\n",
            "\n",
            "options:\n",
            "  -h, --help            show this help message and exit\n",
            "\n"
          ]
        }
      ],
      "source": [
        "%%palm --help"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-9eQhRTTVtum"
      },
      "source": [
        "### Getting a PaLM API key\n",
        "\n",
        "To use the PaLM API, you will need to [create an API key](https://developers.generativeai.google/tutorials/setup). (You only need to do this step once.)\n",
        "\n",
        "### Set the API key in the notebook\n",
        "\n",
        "Set your API key by running the cell below.\n",
        "\n",
        "Caution: The API key is a secret that grants access to the API from your account so make sure to remove it from the notebook before saving, and do not share any notebooks that contain API keys."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qz1S192CUfQ3"
      },
      "outputs": [],
      "source": [
        "%env GOOGLE_API_KEY=YOUR PALM KEY"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d7MCwdX6p4Pd"
      },
      "source": [
        "## PaLM magic commands: `run`, `compile`, `compare`, and `evaluate`\n",
        "\n",
        "PaLM magics provide four different commands:\n",
        "1. `run`\n",
        "1. `compile`\n",
        "1. `compare`\n",
        "1. `evaluate`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7YDLl0i0cYrk"
      },
      "source": [
        "### Command: `palm run`\n",
        "\n",
        "The `run` command sends the contents of the cell to the model.\n",
        "\n",
        "Because running prompts is so common, PaLM magics defaults to the `run` command if no command is given. For example, the next two cells are identical."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CrE6WUhwcxjz"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of hot is\",\n\"cold.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-0c682acf-471c-429d-b311-8d0caed7bdb9\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of hot is</td>\n",
              "      <td>cold.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0c682acf-471c-429d-b311-8d0caed7bdb9')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-691c48fd-ac27-4155-aa4e-f2801e653849\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-691c48fd-ac27-4155-aa4e-f2801e653849')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-691c48fd-ac27-4155-aa4e-f2801e653849 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-691c48fd-ac27-4155-aa4e-f2801e653849');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-0c682acf-471c-429d-b311-8d0caed7bdb9 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-0c682acf-471c-429d-b311-8d0caed7bdb9');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                  Prompt text_result\n",
              "0           0          0           0  The opposite of hot is       cold."
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm run\n",
        "The opposite of hot is"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xzIqZlsbyQJj"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of hot is\",\n\"cold.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-f588912a-bb6b-43df-baeb-021fa65bf25e\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of hot is</td>\n",
              "      <td>cold.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f588912a-bb6b-43df-baeb-021fa65bf25e')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-454ce759-3f06-48f6-a3ef-d390f8c82039\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-454ce759-3f06-48f6-a3ef-d390f8c82039')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-454ce759-3f06-48f6-a3ef-d390f8c82039 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-454ce759-3f06-48f6-a3ef-d390f8c82039');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-f588912a-bb6b-43df-baeb-021fa65bf25e button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-f588912a-bb6b-43df-baeb-021fa65bf25e');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                  Prompt text_result\n",
              "0           0          0           0  The opposite of hot is       cold."
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm\n",
        "The opposite of hot is"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yzXeg4Duehbs"
      },
      "source": [
        "#### Understanding the output\n",
        "\n",
        "The `Prompt` column shows the text that was sent to the model, and the `text_result` column shows the result. The other columns will be introduced as you progress through this guide."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UQ1Xs3BsljOu"
      },
      "source": [
        "### Prompt templates\n",
        "\n",
        "Prompts do not have to be fixed strings. You can inject values into a prompt using template placeholders by using `{curly braces}`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hih5E6IhmDDg"
      },
      "outputs": [],
      "source": [
        "english_words = {\n",
        "    # Each value here (hot, cold) will be substituted in for {word} in the prompt\n",
        "    'word': ['hot', 'cold']\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E8WWpTuRmAb1"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of hot is\",\n\"cold.\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of cold is\",\n\"hot.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-b2d4a605-24f6-4835-b691-4644ee304313\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of hot is</td>\n",
              "      <td>cold.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of cold is</td>\n",
              "      <td>hot.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b2d4a605-24f6-4835-b691-4644ee304313')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-b6f7e4e0-4ef4-4885-896b-8bcbe8f31587\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b6f7e4e0-4ef4-4885-896b-8bcbe8f31587')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-b6f7e4e0-4ef4-4885-896b-8bcbe8f31587 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-b6f7e4e0-4ef4-4885-896b-8bcbe8f31587');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-b2d4a605-24f6-4835-b691-4644ee304313 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-b2d4a605-24f6-4835-b691-4644ee304313');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                   Prompt text_result\n",
              "0           0          0           0   The opposite of hot is       cold.\n",
              "1           0          1           0  The opposite of cold is        hot."
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm --inputs english_words\n",
        "The opposite of {word} is"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7iNjF2YY2vzr"
      },
      "source": [
        "#### Understanding the output\n",
        "\n",
        "The `Input Num` column tracks the index of the input word in the list(s). In\n",
        "these examples, `Input Num` of `0` is `'hot'`, and `1` is `'cold'`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M5mOzS_s1smU"
      },
      "source": [
        "#### Specifying multiple sets of inputs\n",
        "\n",
        "You can also specify multiple sets of inputs at one time."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SPfC1bF-CzKB"
      },
      "outputs": [],
      "source": [
        "extreme_temperatures = {\n",
        "    'word': ['hot', 'cold']\n",
        "}\n",
        "minor_temperatures = {\n",
        "    'word': ['warm', 'chilly']\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Smr0Vs3zC_ub"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of hot is\",\n\"cold.\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of cold is\",\n\"warm.\"],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of warm is\",\n\"cold.\"],\n [{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The opposite of chilly is\",\n\"warm.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-e4464f7d-ee9f-4d4a-b694-d32c22a6e4fe\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of hot is</td>\n",
              "      <td>cold.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of cold is</td>\n",
              "      <td>warm.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of warm is</td>\n",
              "      <td>cold.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>The opposite of chilly is</td>\n",
              "      <td>warm.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e4464f7d-ee9f-4d4a-b694-d32c22a6e4fe')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-bf769e89-db3e-40e1-9e44-06923b86ca60\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-bf769e89-db3e-40e1-9e44-06923b86ca60')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-bf769e89-db3e-40e1-9e44-06923b86ca60 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-bf769e89-db3e-40e1-9e44-06923b86ca60');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-e4464f7d-ee9f-4d4a-b694-d32c22a6e4fe button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-e4464f7d-ee9f-4d4a-b694-d32c22a6e4fe');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                     Prompt text_result\n",
              "0           0          0           0     The opposite of hot is       cold.\n",
              "1           0          1           0    The opposite of cold is       warm.\n",
              "2           0          2           0    The opposite of warm is       cold.\n",
              "3           0          3           0  The opposite of chilly is       warm."
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm --inputs extreme_temperatures minor_temperatures\n",
        "The opposite of {word} is"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pL_D8Lko9Tba"
      },
      "source": [
        "### Reading data from Google Sheets\n",
        "\n",
        "The PaLM magic can also read and write to Google Sheets. You will need to be logged in to access Sheets data. This section focuses on reading data from Sheets; a [later section](#sheets_output) shows how you can write output to a Google Sheet."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UNEyCbFGFPo8"
      },
      "source": [
        "#### Log in and authorize access to Sheets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "DkdoxXqjD2Tm"
      },
      "outputs": [],
      "source": [
        "#@title\n",
        "from google.colab import auth\n",
        "auth.authenticate_user()\n",
        "\n",
        "import google.auth\n",
        "creds, _ = google.auth.default()\n",
        "\n",
        "from google.generativeai.notebook import magics\n",
        "magics.authorize(creds)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xtVfJlG5GudI"
      },
      "source": [
        "#### Formatting a spreadsheet for use with the PaLM magic\n",
        "\n",
        "Pass the ID or URL of a Google Sheet to the `--sheets_input_names` flag to load it up as template data.\n",
        "\n",
        "Use the following format in your spreadsheet to use the data in a prompt template:\n",
        "1. Put the names of the variables (of your prompt template) in the first row of the sheet.\n",
        "1. Put the data to substitute for each variable in the rows below.\n",
        "\n",
        "For example, if your prompt template has two variables to substitute, `name` and `temperament`, you would write your spreadsheet like this:\n",
        "\n",
        "|name|temperament|\n",
        "-----|-----------\n",
        "|Milo|cheeky|\n",
        "|Bigsly|relaxed|\n",
        "|Subra|shy|"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "R2B35-6S7Z3f"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "Reading inputs from worksheet <a target=\"_blank\" rel=\"noopener\" href=\"https://sheets.googleapis.com/v4/spreadsheets/1UHfpkmBqIX5RjeJcGXOevIEhMmEoKlf5f9teqwQyHqc#gid=0\">monkeys in Monkey Magics</a>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Milo and it has a cheeky temperament.\",\n\"Milo the monkey has a cheeky temperament, always getting into mischief and playing pranks on the other animals in the jungle.\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Bigsly and it has a relaxed temperament.\",\n\"Bigsly the monkey is a relaxed, easy-going creature who enjoys spending time in the sun and eating bananas.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-a22349ce-ee9e-449f-b37d-6a2be9a38749\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Milo the monkey has a cheeky temperament, alwa...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Bigsly the monkey is a relaxed, easy-going cre...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a22349ce-ee9e-449f-b37d-6a2be9a38749')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-4f127fd9-9f17-44b7-a4b5-977e99d670b5\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4f127fd9-9f17-44b7-a4b5-977e99d670b5')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-4f127fd9-9f17-44b7-a4b5-977e99d670b5 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-4f127fd9-9f17-44b7-a4b5-977e99d670b5');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-a22349ce-ee9e-449f-b37d-6a2be9a38749 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-a22349ce-ee9e-449f-b37d-6a2be9a38749');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num  \\\n",
              "0           0          0           0   \n",
              "1           0          1           0   \n",
              "\n",
              "                                              Prompt  \\\n",
              "0  Create a single sentence description of a monk...   \n",
              "1  Create a single sentence description of a monk...   \n",
              "\n",
              "                                         text_result  \n",
              "0  Milo the monkey has a cheeky temperament, alwa...  \n",
              "1  Bigsly the monkey is a relaxed, easy-going cre...  "
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm --sheets_input_names https://docs.google.com/spreadsheets/d/1UHfpkmBqIX5RjeJcGXOevIEhMmEoKlf5f9teqwQyHqc/edit\n",
        "Create a single sentence description of a monkey's personality. The monkey's name is {name} and it has a {temperament} temperament."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EkR6KZ3KJyF0"
      },
      "source": [
        "#### Try it yourself!\n",
        "\n",
        "To try this out using your own data, create a [new Sheet](http://sheet.new/) and pass the ID to `--sheets_input_names`. As well as ID and URL, you can also search your sheets by title, e.g. `%%palm --sheets_input_names \"Animal adjectives\"`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8pY7xZABEGbi"
      },
      "source": [
        "#### Combining Sheets inputs with Python inputs\n",
        "\n",
        "Sheets inputs can also be combined with `--inputs`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wWKTRt_lETHx"
      },
      "outputs": [],
      "source": [
        "new_monkeys = {\n",
        "    'name': ['Hackerella'],\n",
        "    'temperament': ['clever'],\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yg3n86cKD7H4"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Hackerella and it has a clever temperament.\",\n\"Hackerella is a curious and intelligent monkey who enjoys solving puzzles.\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Milo and it has a cheeky temperament.\",\n\"Milo is a mischievous little monkey who loves to play and get into trouble.\"],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Bigsly and it has a relaxed temperament.\",\n\"Bigsly the monkey is a laid-back, easygoing soul.\"],\n [{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Subra and it has a shy temperament.\",\n\"Subra is a shy monkey who enjoys being around people but prefers to stay in the background. He is very intelligent and loves to play games and solve puzzles.\"],\n [{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 4,\n            'f': \"4\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Milo and it has a cheeky temperament.\",\n\"Milo the monkey has a cheeky temperament. He is always getting into trouble, but he always manages to charm his way out of it.\"],\n [{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 5,\n            'f': \"5\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Bigsly and it has a relaxed temperament.\",\n\"Bigsly the monkey is a relaxed, laid-back individual who enjoys spending his days lounging in the sun and eating bananas.\"],\n [{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"Create a single sentence description of a monkey's personality. The monkey's name is Subra and it has a shy temperament.\",\n\"Subra is a shy monkey. He is very timid around people and prefers to be left alone. He is also very sensitive to noise and light, so he likes to spend most of his time in quiet, dark places.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-d754ddeb-ca73-4fa5-8821-00c67095b183\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Hackerella is a curious and intelligent monkey...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Milo is a mischievous little monkey who loves ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Bigsly the monkey is a laid-back, easygoing soul.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Subra is a shy monkey who enjoys being around ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Milo the monkey has a cheeky temperament. He i...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Bigsly the monkey is a relaxed, laid-back indi...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>0</td>\n",
              "      <td>6</td>\n",
              "      <td>0</td>\n",
              "      <td>Create a single sentence description of a monk...</td>\n",
              "      <td>Subra is a shy monkey. He is very timid around...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d754ddeb-ca73-4fa5-8821-00c67095b183')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-018058a7-2f38-4a73-b826-ee3cf4517570\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-018058a7-2f38-4a73-b826-ee3cf4517570')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-018058a7-2f38-4a73-b826-ee3cf4517570 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-018058a7-2f38-4a73-b826-ee3cf4517570');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-d754ddeb-ca73-4fa5-8821-00c67095b183 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-d754ddeb-ca73-4fa5-8821-00c67095b183');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num  \\\n",
              "0           0          0           0   \n",
              "1           0          1           0   \n",
              "2           0          2           0   \n",
              "3           0          3           0   \n",
              "4           0          4           0   \n",
              "5           0          5           0   \n",
              "6           0          6           0   \n",
              "\n",
              "                                              Prompt  \\\n",
              "0  Create a single sentence description of a monk...   \n",
              "1  Create a single sentence description of a monk...   \n",
              "2  Create a single sentence description of a monk...   \n",
              "3  Create a single sentence description of a monk...   \n",
              "4  Create a single sentence description of a monk...   \n",
              "5  Create a single sentence description of a monk...   \n",
              "6  Create a single sentence description of a monk...   \n",
              "\n",
              "                                         text_result  \n",
              "0  Hackerella is a curious and intelligent monkey...  \n",
              "1  Milo is a mischievous little monkey who loves ...  \n",
              "2  Bigsly the monkey is a laid-back, easygoing soul.  \n",
              "3  Subra is a shy monkey who enjoys being around ...  \n",
              "4  Milo the monkey has a cheeky temperament. He i...  \n",
              "5  Bigsly the monkey is a relaxed, laid-back indi...  \n",
              "6  Subra is a shy monkey. He is very timid around...  "
            ]
          },
          "execution_count": null,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm --inputs new_monkeys --sheets_input_names 1UHfpkmBqIX5RjeJcGXOevIEhMmEoKlf5f9teqwQyHqc 1UHfpkmBqIX5RjeJcGXOevIEhMmEoKlf5f9teqwQyHqc\n",
        "Create a single sentence description of a monkey's personality. The monkey's name is {name} and it has a {temperament} temperament."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6-PRKeLhUa8w"
      },
      "source": [
        "### Command: `palm eval`\n",
        "\n",
        "Use `%%palm eval` to compare the output of a prompt with known ground-truth data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KeWujB0MUsV6"
      },
      "outputs": [],
      "source": [
        "test_data = {\n",
        "    \"word\": [\"dog\", \"cat\", \"house\"]\n",
        "}\n",
        "ground_truth = [\"chien\", \"chat\", \"maison\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "We_-1C2UU9Mh"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'dog'}\",\n\"chien English: cat\\nFrench: chat\",\n\"chien\",\nfalse],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'cat'}\",\n\"chat English: book\\nFrench: livre\",\n\"chat\",\nfalse],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'house'}\",\n\"maison English: I love you\\nFrench: Je t'aime\",\n\"maison\",\nfalse]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt vars\"], [\"string\", \"actual_text_result\"], [\"string\", \"ground_truth_text_result\"], [\"string\", \"is_equal\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-eb0975f7-428a-4d2b-a9b5-c1bc83c44409\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt vars</th>\n",
              "      <th>actual_text_result</th>\n",
              "      <th>ground_truth_text_result</th>\n",
              "      <th>is_equal</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'dog'}</td>\n",
              "      <td>chien English: cat\\nFrench: chat</td>\n",
              "      <td>chien</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'cat'}</td>\n",
              "      <td>chat English: book\\nFrench: livre</td>\n",
              "      <td>chat</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'house'}</td>\n",
              "      <td>maison English: I love you\\nFrench: Je t'aime</td>\n",
              "      <td>maison</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-eb0975f7-428a-4d2b-a9b5-c1bc83c44409')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-0c3863f8-1c72-4bec-ae77-824b5352b471\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0c3863f8-1c72-4bec-ae77-824b5352b471')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-0c3863f8-1c72-4bec-ae77-824b5352b471 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-0c3863f8-1c72-4bec-ae77-824b5352b471');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-eb0975f7-428a-4d2b-a9b5-c1bc83c44409 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-eb0975f7-428a-4d2b-a9b5-c1bc83c44409');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num        Prompt vars  \\\n",
              "0           0          0           0    {'word': 'dog'}   \n",
              "1           0          1           0    {'word': 'cat'}   \n",
              "2           0          2           0  {'word': 'house'}   \n",
              "\n",
              "                              actual_text_result ground_truth_text_result  \\\n",
              "0               chien English: cat\\nFrench: chat                    chien   \n",
              "1              chat English: book\\nFrench: livre                     chat   \n",
              "2  maison English: I love you\\nFrench: Je t'aime                   maison   \n",
              "\n",
              "   is_equal  \n",
              "0     False  \n",
              "1     False  \n",
              "2     False  "
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm eval --inputs test_data --ground_truth ground_truth\n",
        "English: Hello\n",
        "French: Bonjour\n",
        "English: {word}\n",
        "French:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Z96kyObETyA"
      },
      "source": [
        "#### Post processing model outputs\n",
        "\n",
        "To perform ground-truth testing, you may need to post-process the model output.\n",
        "\n",
        "[Post-processing](#post_processing) functions allow you to define a function that processes the model output. In the case of the `eval` command, only the result column is used in the final equality check.\n",
        "\n",
        "Use the `post_process_replace_fn` decorator to define a function to post-process results:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E7f_h6UgETyA"
      },
      "outputs": [],
      "source": [
        "from google.generativeai.notebook import magics\n",
        "\n",
        "# Define a function to extract only the first response.\n",
        "@magics.post_process_replace_fn\n",
        "def extract_and_normalize(input):\n",
        "  first_line, *unused = input.split('English:')\n",
        "  return first_line.strip().lower()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pFuGfjWiETyA"
      },
      "source": [
        "The `extract_and_normalize` function defined above will take the output from the model and trim any repeated language pairs, leaving just the first response. Check out the [post-processing](#post_processing) section to learn more about post-processing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w7hFcIiMETyA"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'dog'}\",\n\"chien\",\n\"chien\",\ntrue],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'cat'}\",\n\"chat\",\n\"chat\",\ntrue],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'house'}\",\n\"maison\",\n\"maison\",\ntrue]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt vars\"], [\"string\", \"actual_text_result\"], [\"string\", \"ground_truth_text_result\"], [\"string\", \"is_equal\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-4597f4ee-e4b5-4112-ab2d-55865269b3d7\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt vars</th>\n",
              "      <th>actual_text_result</th>\n",
              "      <th>ground_truth_text_result</th>\n",
              "      <th>is_equal</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'dog'}</td>\n",
              "      <td>chien</td>\n",
              "      <td>chien</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'cat'}</td>\n",
              "      <td>chat</td>\n",
              "      <td>chat</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'house'}</td>\n",
              "      <td>maison</td>\n",
              "      <td>maison</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4597f4ee-e4b5-4112-ab2d-55865269b3d7')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-e399fc70-561e-4046-af43-5ba4c6d6b6e7\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e399fc70-561e-4046-af43-5ba4c6d6b6e7')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-e399fc70-561e-4046-af43-5ba4c6d6b6e7 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-e399fc70-561e-4046-af43-5ba4c6d6b6e7');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-4597f4ee-e4b5-4112-ab2d-55865269b3d7 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-4597f4ee-e4b5-4112-ab2d-55865269b3d7');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num        Prompt vars actual_text_result  \\\n",
              "0           0          0           0    {'word': 'dog'}              chien   \n",
              "1           0          1           0    {'word': 'cat'}               chat   \n",
              "2           0          2           0  {'word': 'house'}             maison   \n",
              "\n",
              "  ground_truth_text_result  is_equal  \n",
              "0                    chien      True  \n",
              "1                     chat      True  \n",
              "2                   maison      True  "
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm eval --inputs test_data --ground_truth ground_truth | extract_and_normalize\n",
        "English: Hello\n",
        "French: Bonjour\n",
        "English: {word}\n",
        "French:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C60tDBCnDUOl"
      },
      "source": [
        "### Command: `palm compile`\n",
        "\n",
        "Use the `%%palm compile` command to convert a prompt with placeholders to a  function callable from within Python.\n",
        "\n",
        "All flags and post-processing are \"compiled\" into the function and will be used when invoked.\n",
        "\n",
        "In this example, a function called `translate_en_to_fr` is created, using the `extract_and_normalize` post-processing function from [before](#palm_eval)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QDXqCknx_AsY"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Saved function to Python variable: translate_en_to_fr'"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm compile translate_en_to_fr | extract_and_normalize\n",
        "English: Hello\n",
        "French: Bonjour\n",
        "English: {word}\n",
        "French:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-Ax3vb9r_pLD"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"English: Hello\\nFrench: Bonjour\\nEnglish: cat\\nFrench:\",\n\"chat\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"English: Hello\\nFrench: Bonjour\\nEnglish: dog\\nFrench:\",\n\"chien\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-4e4890bc-2963-4472-b00e-8c54a1389642\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>English: Hello\\nFrench: Bonjour\\nEnglish: cat\\...</td>\n",
              "      <td>chat</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>English: Hello\\nFrench: Bonjour\\nEnglish: dog\\...</td>\n",
              "      <td>chien</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4e4890bc-2963-4472-b00e-8c54a1389642')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-e99c28c7-2f87-4e04-b8a6-ba1e82833815\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e99c28c7-2f87-4e04-b8a6-ba1e82833815')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-e99c28c7-2f87-4e04-b8a6-ba1e82833815 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-e99c28c7-2f87-4e04-b8a6-ba1e82833815');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-4e4890bc-2963-4472-b00e-8c54a1389642 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-4e4890bc-2963-4472-b00e-8c54a1389642');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num  \\\n",
              "0           0          0           0   \n",
              "1           0          1           0   \n",
              "\n",
              "                                              Prompt text_result  \n",
              "0  English: Hello\\nFrench: Bonjour\\nEnglish: cat\\...        chat  \n",
              "1  English: Hello\\nFrench: Bonjour\\nEnglish: dog\\...       chien  "
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "en_words = ['cat', 'dog']\n",
        "translate_en_to_fr({'word': en_words})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kgnqN_QAF75T"
      },
      "source": [
        "#### Output formats\n",
        "\n",
        "By default, a \"compiled\" function returns its output as an object that will be displayed as Pandas `DataFrame`. However, you can convert the results object to a `DataFrame` or dictionary with `.as_dict()` or `.as_dataframe()`, respectively.\n",
        "\n",
        "For more information, see the [`--outputs`](#python_output) flag."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X-80vOvMBaUr"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "chat is French for cat\n",
            "chien is French for dog\n"
          ]
        }
      ],
      "source": [
        "results = translate_en_to_fr({'word': en_words}).as_dict()\n",
        "\n",
        "fr_words = results['text_result']\n",
        "\n",
        "for en, fr in zip(en_words, fr_words):\n",
        "  print(f'{fr} is French for {en}')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lxZ__1p9C5Xq"
      },
      "source": [
        "### Command: `palm compare`\n",
        "\n",
        "`%%palm compare` runs compiled prompts and produces a table with the comparison results side-by-side, so you can inspect the differences."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lTnp0cAidtGe"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Saved function to Python variable: few_shot_prompt'"
            ]
          },
          "execution_count": 30,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm compile few_shot_prompt\n",
        "English: Hello\n",
        "French: Bonjour\n",
        "English: {word}\n",
        "French:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zpVAcNeteAMh"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Saved function to Python variable: zero_shot_prompt'"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm compile zero_shot_prompt\n",
        "{word} translated to French is:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mKVVnRxRyzh8"
      },
      "outputs": [],
      "source": [
        "words = {\n",
        "    \"word\": [\"dog\", \"cat\", \"house\"]\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gv1EFjFWeJNG"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'dog'}\",\n\"chien\\nEnglish: cat\\nFrench: chat\\nEnglish: I love you\\nFrench: Je t'aime\",\n\"Dog is \\\"chien\\\" in French.\",\nfalse],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'cat'}\",\n\"Chat\\nEnglish: dog\\nFrench: Chien\\nEnglish: house\\nFrench: Maison\\nEnglish: car\\nFrench: Voiture\",\n\"Chat\",\nfalse],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'house'}\",\n\"maison English: car\\nFrench: voiture\",\n\"Maison\",\nfalse]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt vars\"], [\"string\", \"few_shot_prompt_text_result\"], [\"string\", \"zero_shot_prompt_text_result\"], [\"string\", \"is_equal\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-cb86692c-2e55-496d-a9a6-a2473edfbe85\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt vars</th>\n",
              "      <th>few_shot_prompt_text_result</th>\n",
              "      <th>zero_shot_prompt_text_result</th>\n",
              "      <th>is_equal</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'dog'}</td>\n",
              "      <td>chien\\nEnglish: cat\\nFrench: chat\\nEnglish: I ...</td>\n",
              "      <td>Dog is \"chien\" in French.</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'cat'}</td>\n",
              "      <td>Chat\\nEnglish: dog\\nFrench: Chien\\nEnglish: ho...</td>\n",
              "      <td>Chat</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'house'}</td>\n",
              "      <td>maison English: car\\nFrench: voiture</td>\n",
              "      <td>Maison</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cb86692c-2e55-496d-a9a6-a2473edfbe85')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-91ec4581-7159-4315-8f29-66239919745f\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-91ec4581-7159-4315-8f29-66239919745f')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-91ec4581-7159-4315-8f29-66239919745f button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-91ec4581-7159-4315-8f29-66239919745f');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-cb86692c-2e55-496d-a9a6-a2473edfbe85 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-cb86692c-2e55-496d-a9a6-a2473edfbe85');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num        Prompt vars  \\\n",
              "0           0          0           0    {'word': 'dog'}   \n",
              "1           0          1           0    {'word': 'cat'}   \n",
              "2           0          2           0  {'word': 'house'}   \n",
              "\n",
              "                         few_shot_prompt_text_result  \\\n",
              "0  chien\\nEnglish: cat\\nFrench: chat\\nEnglish: I ...   \n",
              "1  Chat\\nEnglish: dog\\nFrench: Chien\\nEnglish: ho...   \n",
              "2               maison English: car\\nFrench: voiture   \n",
              "\n",
              "  zero_shot_prompt_text_result  is_equal  \n",
              "0    Dog is \"chien\" in French.     False  \n",
              "1                         Chat     False  \n",
              "2                       Maison     False  "
            ]
          },
          "execution_count": 31,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm compare few_shot_prompt zero_shot_prompt --inputs words"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fFOpgeC2Idzl"
      },
      "source": [
        "#### Custom comparison functions\n",
        "\n",
        "By default, `compare` just checks for equalilty in the returned results. However, you can specify one or more custom functions with the `--compare_fn` flag:."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bd9eDrxvHzS5"
      },
      "outputs": [],
      "source": [
        "def average_word_length(lhs, rhs):\n",
        "  \"\"\"Count the average number of words used across prompts.\"\"\"\n",
        "  return (len(lhs.split(' ')) + len(rhs.split(' '))) / 2\n",
        "\n",
        "def shortest_answer(lhs, rhs):\n",
        "  \"\"\"Label the prompt that generated the shortest output.\"\"\"\n",
        "  if len(lhs) < len(rhs):\n",
        "    return 'first'\n",
        "  elif len(lhs) > len(rhs):\n",
        "    return 'second'\n",
        "  else:\n",
        "    return 'same'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UKBLkZ6kHncJ"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'dog'}\",\n\"chien\\nEnglish: cat\\nFrench: chat\",\n\"The French word for dog is chien.\",\n{\n            'v': 5.0,\n            'f': \"5.0\",\n        },\n\"first\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'cat'}\",\n\"chat English: car\\nFrench: voiture\",\n\"\\\"cat\\\" is \\\"chat\\\" in French\",\n{\n            'v': 4.5,\n            'f': \"4.5\",\n        },\n\"second\"],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"{'word': 'house'}\",\n\"maison\",\n\"maison\",\n{\n            'v': 1.0,\n            'f': \"1.0\",\n        },\n\"same\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt vars\"], [\"string\", \"few_shot_prompt_text_result\"], [\"string\", \"zero_shot_prompt_text_result\"], [\"number\", \"average_word_length\"], [\"string\", \"shortest_answer\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-ece06d6a-9857-4f5a-bdf4-86145ec06a6b\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt vars</th>\n",
              "      <th>few_shot_prompt_text_result</th>\n",
              "      <th>zero_shot_prompt_text_result</th>\n",
              "      <th>average_word_length</th>\n",
              "      <th>shortest_answer</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'dog'}</td>\n",
              "      <td>chien\\nEnglish: cat\\nFrench: chat</td>\n",
              "      <td>The French word for dog is chien.</td>\n",
              "      <td>5.0</td>\n",
              "      <td>first</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'cat'}</td>\n",
              "      <td>chat English: car\\nFrench: voiture</td>\n",
              "      <td>\"cat\" is \"chat\" in French</td>\n",
              "      <td>4.5</td>\n",
              "      <td>second</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>{'word': 'house'}</td>\n",
              "      <td>maison</td>\n",
              "      <td>maison</td>\n",
              "      <td>1.0</td>\n",
              "      <td>same</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ece06d6a-9857-4f5a-bdf4-86145ec06a6b')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-cdb39172-d011-42e9-9872-273cc644b93e\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-cdb39172-d011-42e9-9872-273cc644b93e')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-cdb39172-d011-42e9-9872-273cc644b93e button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-cdb39172-d011-42e9-9872-273cc644b93e');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-ece06d6a-9857-4f5a-bdf4-86145ec06a6b button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-ece06d6a-9857-4f5a-bdf4-86145ec06a6b');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num        Prompt vars  \\\n",
              "0           0          0           0    {'word': 'dog'}   \n",
              "1           0          1           0    {'word': 'cat'}   \n",
              "2           0          2           0  {'word': 'house'}   \n",
              "\n",
              "          few_shot_prompt_text_result       zero_shot_prompt_text_result  \\\n",
              "0   chien\\nEnglish: cat\\nFrench: chat  The French word for dog is chien.   \n",
              "1  chat English: car\\nFrench: voiture          \"cat\" is \"chat\" in French   \n",
              "2                              maison                             maison   \n",
              "\n",
              "   average_word_length shortest_answer  \n",
              "0                  5.0           first  \n",
              "1                  4.5          second  \n",
              "2                  1.0            same  "
            ]
          },
          "execution_count": 33,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm compare few_shot_prompt zero_shot_prompt --inputs words --compare_fn average_word_length shortest_answer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gpygJDRZz-gf"
      },
      "source": [
        "## Other commands"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K_Yc3EG8X8mF"
      },
      "source": [
        "### Help\n",
        "\n",
        "The `--help` flag displays the supported commands that you can pass directly to `%%palm`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8NOauYgPYh62"
      },
      "source": [
        "Append `--help` to view detailed documentation for each command. For example,"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gfKb33diYSHn"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "usage: palm run [-h] [--model_type {echo,text}] [--temperature TEMPERATURE]\n",
            "                [--model MODEL] [--candidate_count CANDIDATE_COUNT] [--unique]\n",
            "                [--inputs INPUTS [INPUTS ...]]\n",
            "                [--sheets_input_names SHEETS_INPUT_NAMES [SHEETS_INPUT_NAMES ...]]\n",
            "                [--outputs OUTPUTS [OUTPUTS ...]]\n",
            "                [--sheets_output_names SHEETS_OUTPUT_NAMES [SHEETS_OUTPUT_NAMES ...]]\n",
            "\n",
            "options:\n",
            "  -h, --help            show this help message and exit\n",
            "  --model_type {echo,text}, -mt {echo,text}\n",
            "                        The type of model to use.\n",
            "  --temperature TEMPERATURE, -t TEMPERATURE\n",
            "                        Controls the randomness of the output. Must be\n",
            "                        positive. Typical values are in the range: [0.0, 1.0].\n",
            "                        Higher values produce a more random and varied\n",
            "                        response. A temperature of zero will be deterministic.\n",
            "  --model MODEL, -m MODEL\n",
            "                        The name of the model to use. If not provided, a\n",
            "                        default model will be used.\n",
            "  --candidate_count CANDIDATE_COUNT, -cc CANDIDATE_COUNT\n",
            "                        The number of candidates to produce.\n",
            "  --unique              Whether to dedupe candidates returned by the model.\n",
            "  --inputs INPUTS [INPUTS ...], -i INPUTS [INPUTS ...]\n",
            "                        Optional names of Python variables containing inputs\n",
            "                        to use to instantiate a prompt. The variable must be\n",
            "                        either: a dictionary {'key1': ['val1', 'val2'] ...},\n",
            "                        or an instance of LLMFnInputsSource such as\n",
            "                        SheetsInput.\n",
            "  --sheets_input_names SHEETS_INPUT_NAMES [SHEETS_INPUT_NAMES ...], -si SHEETS_INPUT_NAMES [SHEETS_INPUT_NAMES ...]\n",
            "                        Optional names of Google Sheets to read inputs from.\n",
            "                        This is equivalent to using --inputs with the names of\n",
            "                        variables that are instances of SheetsInputs, just\n",
            "                        more convenient to use.\n",
            "  --outputs OUTPUTS [OUTPUTS ...], -o OUTPUTS [OUTPUTS ...]\n",
            "                        Optional names of Python variables to output to. If\n",
            "                        the Python variable has not already been defined, it\n",
            "                        will be created. If the variable is defined and is an\n",
            "                        instance of LLMFnOutputsSink, the outputs will be\n",
            "                        written through the sink's write_outputs() method.\n",
            "  --sheets_output_names SHEETS_OUTPUT_NAMES [SHEETS_OUTPUT_NAMES ...], -so SHEETS_OUTPUT_NAMES [SHEETS_OUTPUT_NAMES ...]\n",
            "                        Optional names of Google Sheets to write inputs to.\n",
            "                        This is equivalent to using --outputs with the names\n",
            "                        of variables that are instances of SheetsOutputs, just\n",
            "                        more convenient to use.\n",
            "\n"
          ]
        }
      ],
      "source": [
        "%%palm run --help"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lKFZ7h3VfFYg"
      },
      "source": [
        "### Models\n",
        "\n",
        "Use the `--model` flag to specify the PaLM model variant you wish to use.\n",
        "\n",
        "See the [`list_models()`](/api/python/google/generativeai/list_models) method to retrieve the supported models. The PaLM magic can be used with any model supporting the `generateText` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NmD8zg1lhrRF"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"My favourite color is\",\n\"green The color green is associated with many things, including nature, growth, and health. It is also often seen as a symbol of hope and renewal. Green is a versatile color that can be used in many different ways, and it can be both calming and energizing.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-c3abf194-d46d-467a-b43e-f95ce22e29aa\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>My favourite color is</td>\n",
              "      <td>green The color green is associated with many ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c3abf194-d46d-467a-b43e-f95ce22e29aa')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-88e75049-819f-4631-a7a5-b7fa477438f0\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-88e75049-819f-4631-a7a5-b7fa477438f0')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-88e75049-819f-4631-a7a5-b7fa477438f0 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-88e75049-819f-4631-a7a5-b7fa477438f0');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-c3abf194-d46d-467a-b43e-f95ce22e29aa button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-c3abf194-d46d-467a-b43e-f95ce22e29aa');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                 Prompt  \\\n",
              "0           0          0           0  My favourite color is   \n",
              "\n",
              "                                         text_result  \n",
              "0  green The color green is associated with many ...  "
            ]
          },
          "execution_count": 35,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm run --model models/text-bison-001\n",
        "My favourite color is"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9oSaEgkIiEqC"
      },
      "source": [
        "#### Model parameters\n",
        "\n",
        "You can also configure model parameters, such as [`--candidate_count`](#candidate_count) and `--temperature`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FB5Rswg8iPfm"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"My favourite color is\",\n\"blue. I love the way it looks on me, and I love the way it makes me feel. Blue is a calming and relaxing color, and it can help to create a sense of peace and tranquility. I also love the way blue can be used to create a sense of depth and dimension. When used correctly, blue can make a room feel larger and more spacious.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-bd47933f-9985-447d-aad7-9eb5b4d69383\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>My favourite color is</td>\n",
              "      <td>blue. I love the way it looks on me, and I lov...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bd47933f-9985-447d-aad7-9eb5b4d69383')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-ff94d6ce-b39d-4aaa-b216-5d92f4dfc248\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ff94d6ce-b39d-4aaa-b216-5d92f4dfc248')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-ff94d6ce-b39d-4aaa-b216-5d92f4dfc248 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-ff94d6ce-b39d-4aaa-b216-5d92f4dfc248');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-bd47933f-9985-447d-aad7-9eb5b4d69383 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-bd47933f-9985-447d-aad7-9eb5b4d69383');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                 Prompt  \\\n",
              "0           0          0           0  My favourite color is   \n",
              "\n",
              "                                         text_result  \n",
              "0  blue. I love the way it looks on me, and I lov...  "
            ]
          },
          "execution_count": 36,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm run --model models/text-bison-001 --temperature 0.5\n",
        "My favourite color is"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N_MuqgOyikAB"
      },
      "source": [
        "### Debugging: The echo model\n",
        "\n",
        "An `echo` model is also available that will echo the prompt back to you. It does not make any API calls or consume your quota so it can be a fast and simple way to test output or post-processing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wq6EX_oki01u"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"A duck's quack does not echo.\",\n\"A duck's quack does not echo.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-f68483c2-32e6-4708-99d4-3f28bc820c13\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>A duck's quack does not echo.</td>\n",
              "      <td>A duck's quack does not echo.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f68483c2-32e6-4708-99d4-3f28bc820c13')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-f94a314a-a68a-4f5d-b5e5-0203e4e1ff49\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f94a314a-a68a-4f5d-b5e5-0203e4e1ff49')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-f94a314a-a68a-4f5d-b5e5-0203e4e1ff49 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-f94a314a-a68a-4f5d-b5e5-0203e4e1ff49');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-f68483c2-32e6-4708-99d4-3f28bc820c13 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-f68483c2-32e6-4708-99d4-3f28bc820c13');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                         Prompt  \\\n",
              "0           0          0           0  A duck's quack does not echo.   \n",
              "\n",
              "                     text_result  \n",
              "0  A duck's quack does not echo.  "
            ]
          },
          "execution_count": 37,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm --model_type echo\n",
        "A duck's quack does not echo."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "18L-_dCAjDWZ"
      },
      "source": [
        "### Export output to Python {:#python_output}\n",
        "\n",
        "In addition to displaying tabular output, the PaLM magic can save model output to Python variables, allowing you to manipulate them further or to export your results.\n",
        "\n",
        "In this example, the output is saved to a Python variable: `fave_colors`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZWAuhyMAjilc"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The best colors to wear in spring-time are\",\n\"* Pastels: These soft, muted colors are perfect for the springtime, as they are fresh and airy. Some popular pastel colors include baby blue, mint green, and pale pink.\\n* Brights: If you want to make a statement, bright colors are a great option for spring. Some popular bright colors include fuchsia, cobalt blue, and yellow.\\n* Neutrals: Neutral colors are always a good choice, as they can be easily dressed up or down. Some popular neutrals include beige, gray, and white.\\n\\nWhen choosing colors to wear in the spring, it is important to consider the occasion and your personal style. For example, if you are attending a formal event, you may want to choose a more muted color palette, such as pastels or neutrals. If you are going for a more casual look, you may want to choose brighter colors, such as brights or pastels.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-33429b76-effa-4dee-81c0-141ad1c89148\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>The best colors to wear in spring-time are</td>\n",
              "      <td>* Pastels: These soft, muted colors are perfec...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-33429b76-effa-4dee-81c0-141ad1c89148')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-ecf01053-7a9a-484b-8002-439553c77bff\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ecf01053-7a9a-484b-8002-439553c77bff')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-ecf01053-7a9a-484b-8002-439553c77bff button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-ecf01053-7a9a-484b-8002-439553c77bff');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-33429b76-effa-4dee-81c0-141ad1c89148 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-33429b76-effa-4dee-81c0-141ad1c89148');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num  \\\n",
              "0           0          0           0   \n",
              "\n",
              "                                       Prompt  \\\n",
              "0  The best colors to wear in spring-time are   \n",
              "\n",
              "                                         text_result  \n",
              "0  * Pastels: These soft, muted colors are perfec...  "
            ]
          },
          "execution_count": 38,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm --outputs fave_colors\n",
        "The best colors to wear in spring-time are"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JMR21trMkACl"
      },
      "source": [
        "Output variables are custom objects that will display as Pandas `DataFrame`s by default. They can be coerced into a Python dictionary or dataframe explicitly by calling `as_dict()` or `as_pandas_dataframe()`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X-pbyRxfj6gB"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'Input Num': [0],\n",
            " 'Prompt': ['The best colors to wear in spring-time are'],\n",
            " 'Prompt Num': [0],\n",
            " 'Result Num': [0],\n",
            " 'text_result': ['* Pastels: These soft, muted colors are perfect for the '\n",
            "                 'springtime, as they are fresh and airy. Some popular pastel '\n",
            "                 'colors include baby blue, mint green, and pale pink.\\n'\n",
            "                 '* Brights: If you want to make a statement, bright colors '\n",
            "                 'are a great option for spring. Some popular bright colors '\n",
            "                 'include fuchsia, cobalt blue, and yellow.\\n'\n",
            "                 '* Neutrals: Neutral colors are always a good choice, as they '\n",
            "                 'can be easily dressed up or down. Some popular neutrals '\n",
            "                 'include beige, gray, and white.\\n'\n",
            "                 '\\n'\n",
            "                 'When choosing colors to wear in the spring, it is important '\n",
            "                 'to consider the occasion and your personal style. For '\n",
            "                 'example, if you are attending a formal event, you may want '\n",
            "                 'to choose a more muted color palette, such as pastels or '\n",
            "                 'neutrals. If you are going for a more casual look, you may '\n",
            "                 'want to choose brighter colors, such as brights or pastels.']}\n"
          ]
        }
      ],
      "source": [
        "from pprint import pprint\n",
        "\n",
        "pprint(fave_colors.as_dict())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7hI7bQVaC4gc"
      },
      "source": [
        "### Write to Google Sheets {:#sheets_output}\n",
        "\n",
        "You can save output back to Google Sheets, using `--sheets_output_names`. You must be logged in, and you must have the appropriate permissions to access private Sheets.\n",
        "\n",
        "To try this out, create a [new Sheet](http://sheet.new/) and name it `Translation results`. Like the input flag, the `--sheets_output_names` flag also accepts the sheet URL or ID in place of the textual name."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3AlFnpnj_ICo"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"English: Hello\\nFrench: Bonjour\\nEnglish: dog\\nFrench:\",\n\"chien\\nEnglish: cat\\nFrench:  chat\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"English: Hello\\nFrench: Bonjour\\nEnglish: cat\\nFrench:\",\n\"chat English: how many ?\\nFrench: Combien ?\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-e1dde0f0-f8e5-465b-8b14-3549194422ea\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>English: Hello\\nFrench: Bonjour\\nEnglish: dog\\...</td>\n",
              "      <td>chien\\nEnglish: cat\\nFrench:  chat</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>English: Hello\\nFrench: Bonjour\\nEnglish: cat\\...</td>\n",
              "      <td>chat English: how many ?\\nFrench: Combien ?</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e1dde0f0-f8e5-465b-8b14-3549194422ea')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-7a104fb3-c3bd-4a3f-a313-17b195f58296\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-7a104fb3-c3bd-4a3f-a313-17b195f58296')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-7a104fb3-c3bd-4a3f-a313-17b195f58296 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-7a104fb3-c3bd-4a3f-a313-17b195f58296');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-e1dde0f0-f8e5-465b-8b14-3549194422ea button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-e1dde0f0-f8e5-465b-8b14-3549194422ea');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num  \\\n",
              "0           0          0           0   \n",
              "1           0          1           0   \n",
              "\n",
              "                                              Prompt  \\\n",
              "0  English: Hello\\nFrench: Bonjour\\nEnglish: dog\\...   \n",
              "1  English: Hello\\nFrench: Bonjour\\nEnglish: cat\\...   \n",
              "\n",
              "                                   text_result  \n",
              "0           chien\\nEnglish: cat\\nFrench:  chat  \n",
              "1  chat English: how many ?\\nFrench: Combien ?  "
            ]
          },
          "execution_count": null,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm --inputs english_words --sheets_output_names \"Translation results\"\n",
        "English: Hello\n",
        "French: Bonjour\n",
        "English: {word}\n",
        "French:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WLOC9jNC_Q5K"
      },
      "source": [
        "The results are saved to a new tab and contain the same data you see here in Colab.\n",
        "\n",
        "![Example of a saved sheet](https://developers.generativeai.google/tools/sheets_output.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NlfVPS6C1Pwn"
      },
      "source": [
        "### Generating multiple candidates {:#candidate_count}\n",
        "\n",
        "To generate more than one output for a single prompt, you can pass `--candidate_count` to the model. This is set to 1 by default, which outputs only the top result.\n",
        "\n",
        "Sometimes the model will generate the same output across candidates. These can be filtered with the `--unique` flag, which de-duplicates results out of the candidate batch (but not across multiple prompts)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "y5YMJeQH4EnY"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"In a single word, my favourite color is\",\n\"cerulean\"],\n [{\n            'v': 1,\n            'f': \"1\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 1,\n            'f': \"1\",\n        },\n\"In a single word, my favourite color is\",\n\"Magenta.\"],\n [{\n            'v': 2,\n            'f': \"2\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 2,\n            'f': \"2\",\n        },\n\"In a single word, my favourite color is\",\n\"**blue**.\"],\n [{\n            'v': 3,\n            'f': \"3\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 6,\n            'f': \"6\",\n        },\n\"In a single word, my favourite color is\",\n\"magenta.\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-5ad4575f-3f20-4856-9d11-fe2ae83cb9de\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>In a single word, my favourite color is</td>\n",
              "      <td>cerulean</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>In a single word, my favourite color is</td>\n",
              "      <td>Magenta.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>In a single word, my favourite color is</td>\n",
              "      <td>**blue**.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>6</td>\n",
              "      <td>In a single word, my favourite color is</td>\n",
              "      <td>magenta.</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5ad4575f-3f20-4856-9d11-fe2ae83cb9de')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-fc9cb591-71dd-49d0-b0f5-ee5a62b91fcc\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-fc9cb591-71dd-49d0-b0f5-ee5a62b91fcc')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-fc9cb591-71dd-49d0-b0f5-ee5a62b91fcc button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-fc9cb591-71dd-49d0-b0f5-ee5a62b91fcc');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-5ad4575f-3f20-4856-9d11-fe2ae83cb9de button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-5ad4575f-3f20-4856-9d11-fe2ae83cb9de');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                                   Prompt  \\\n",
              "0           0          0           0  In a single word, my favourite color is   \n",
              "1           0          0           1  In a single word, my favourite color is   \n",
              "2           0          0           2  In a single word, my favourite color is   \n",
              "3           0          0           6  In a single word, my favourite color is   \n",
              "\n",
              "  text_result  \n",
              "0    cerulean  \n",
              "1    Magenta.  \n",
              "2   **blue**.  \n",
              "3    magenta.  "
            ]
          },
          "execution_count": null,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm run --temperature 1.0 --candidate_count 8 --unique\n",
        "In a single word, my favourite color is"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1PukLL8GGKTJ"
      },
      "source": [
        "The `Result Num` column distinguishes multiple candidates generated from the same prompt."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "47cshlf2DgBe"
      },
      "source": [
        "### Post-processing model output {:#post_processing}\n",
        "\n",
        "The broad range of possible outputs and structures can make it difficult to adapt the model's output to your problem domain. The PaLM magic provides post-processing options that allow you to modify or process model output using Python code.\n",
        "\n",
        "Post-processing functions can either add a new column to the output, or modify the `text_result` column. The `text_result` column is the last column, and is used by the `eval` and `compare` commands to determine the final output.\n",
        "\n",
        "Here are some sample functions to use in post-processing. One adds a new column and the other updates the result column, using the `post_process_replace_fn` decorator."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KA94swwJKWjC"
      },
      "outputs": [],
      "source": [
        "import re\n",
        "from google.generativeai.notebook import magics\n",
        "\n",
        "# Add a new column.\n",
        "def word_count(result):\n",
        "  return len(result.split(' '))\n",
        "\n",
        "# Modify the text_result column\n",
        "@magics.post_process_replace_fn\n",
        "def extract_first_sentence(result):\n",
        "  \"\"\"Extracts the first word from the raw result.\"\"\"\n",
        "  first, *_ = re.split(r'\\.\\s*', result)\n",
        "  return first"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7zxqnoBi7KVQ"
      },
      "source": [
        "To use these functions, append them to the `%%palm` command using the pipe (`|`) operator, like so."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "q1ZusKr7KTBo"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.module+javascript": "\n      import \"https://ssl.gstatic.com/colaboratory/data_table/99dac6621f6ae8c4/data_table.js\";\n\n      window.createDataTable({\n        data: [[{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n{\n            'v': 0,\n            'f': \"0\",\n        },\n\"The happiest thing I can imagine is\",\n{\n            'v': 64,\n            'f': \"64\",\n        },\n\"The happiest thing I can imagine is cuddling up with my dog on a cold winter night\"]],\n        columns: [[\"number\", \"index\"], [\"number\", \"Prompt Num\"], [\"number\", \"Input Num\"], [\"number\", \"Result Num\"], [\"string\", \"Prompt\"], [\"number\", \"word_count\"], [\"string\", \"text_result\"]],\n        columnOptions: [{\"width\": \"1px\", \"className\": \"index_column\"}],\n        rowsPerPage: 25,\n        helpUrl: \"https://colab.research.google.com/notebooks/data_table.ipynb\",\n        suppressOutputScrolling: true,\n        minimumWidth: undefined,\n      });\n    ",
            "text/html": [
              "\n",
              "  <div id=\"df-aecce4b4-697a-4e1a-b3a1-fab785b5e00f\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Prompt Num</th>\n",
              "      <th>Input Num</th>\n",
              "      <th>Result Num</th>\n",
              "      <th>Prompt</th>\n",
              "      <th>word_count</th>\n",
              "      <th>text_result</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>The happiest thing I can imagine is</td>\n",
              "      <td>64</td>\n",
              "      <td>The happiest thing I can imagine is cuddling u...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-aecce4b4-697a-4e1a-b3a1-fab785b5e00f')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  \n",
              "    <div id=\"df-c6e0a8be-7329-48d1-b752-3bc4b54882a9\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c6e0a8be-7329-48d1-b752-3bc4b54882a9')\"\n",
              "              title=\"Generate charts.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "      <g>\n",
              "          <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "      </g>\n",
              "  </svg>\n",
              "      </button>\n",
              "    </div>\n",
              "    \n",
              "  <style>\n",
              "    .colab-df-quickchart {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-quickchart:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-quickchart:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "\n",
              "    .colab-quickchart-section-title {\n",
              "        clear: both;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#df-c6e0a8be-7329-48d1-b752-3bc4b54882a9 button.colab-df-quickchart');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#df-c6e0a8be-7329-48d1-b752-3bc4b54882a9');\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'generateCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-aecce4b4-697a-4e1a-b3a1-fab785b5e00f button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-aecce4b4-697a-4e1a-b3a1-fab785b5e00f');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "   Prompt Num  Input Num  Result Num                               Prompt  \\\n",
              "0           0          0           0  The happiest thing I can imagine is   \n",
              "\n",
              "   word_count                                        text_result  \n",
              "0          64  The happiest thing I can imagine is cuddling u...  "
            ]
          },
          "execution_count": null,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "%%palm run | word_count | extract_first_sentence\n",
        "The happiest thing I can imagine is"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XUqf1LWEMaS2"
      },
      "source": [
        "Order matters here. When `word_count` is invoked, the original model output is used to calculate the number of words. If you swap these around, the word count would instead be the number of words in the extracted first sentence."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "stVLo_aT18qN"
      },
      "source": [
        "## Further reading\n",
        "\n",
        "* Refer to the [LLMs concepts guide](https://developers.generativeai.google/guide/concepts) to learn more about LLMs.\n",
        "* Check out the [prompt guidelines](https://developers.generativeai.google/guide/prompt_best_practices) to learn more about crafting prompts to get the most out of working with PaLM.\n",
        "* To prototype and experiment with different prompts, check out [Google AI Studio](https://makersuite.google.com/){:.external}. Also, refer to the [Google AI Studio quickstart](../tutorials/ai-studio_quickstart) for more information."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "notebook_magic.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
